Orthogonal least squares based fast feature selection for linear classification

An Orthogonal Least Squares (OLS) based feature selection method is proposed for both binomial and multinomial classification. The novel Squared Correlation Coefficient (SOCC) defined on Error Reduction Ratio (ERR) in OLS used as the ranking criterion. equivalence between canonical correlation coefficient, Fisher’s criterion, sum of SOCCs revealed, which unveils statistical implication ERR first time. It also shown that has speed advantages when applied greedy search. comprehensively compared with mutual information methods embedded using synthetic real world datasets. results show always top 5 among 12 candidate methods. Besides, can be directly to continuous features without discretisation, another significant advantage over